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Office of Scientific Intelligence 


First Progress Report to Operations Coordinating Board 

by 

Broadcast Evaluation Section 
established under paragraph 8a of 
NSC -169 


I. Summary 

A. The significance of the technical effectiveness of U*S. 
controlled international broadcasting to the national security and 
the importance of determining the degree of effectiveness for policy- 
“—J* 2“?° 8 2L h ?? been recognized at the government level and is 
f n KSC 'l69, Paragraph 8 (a). The problem of determining 
technical (signal) effectiveness of broadcasting within areas to 
^ ce ! s i f generally denied or at least limited and against 
which broadcast jamming and related activities are directed is 
unprecedented. Methods and techniques used in making such evalua- 
tions of commercial broadcasting are entirely inapplicable to this 
problem. New methods and new approaches have had to be evolved and 
are atil! in the process of evolution. 

®? e Broadcast Evaluation Section was established under 

P 5 r S r f Ph ® {a 2' NSC ” 1 ^» h y CIA in conjunction with the Department 
of State, the Department of Defense, and USIA, and under the 
coordination of the Operations Coordinating Board, for intelligence 
SMaaSiSa Of information concerning the technical effectiveness’ of 
U.So International Broadcasting, particularly to the Soviet Bloc. 

.. purpose of this report is to provide information on 

* h ! J^ivities of the Broadcast Evaluation Section, CIA, since its 
establishment in .January 1^4= to provide information on problems 
encountered and studies undertaken with respect thereto; to provide 
information of assistance in interpreting statistical data presented 
in reports of the Broadcast Evaluation Section; to appraise future 
possibilities and limitations of the task; and to provide a general 

estimate of the technical effectiveness of U.S. international broad- 
casting (See Part II). 

i 

®*® organization and staff pattern of the Broadcast 
Evaluation Section was planned by CIA with the advice of USIA and 
the Departments of State and Defense. 
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E. A Technical Evaluation Advisory Committee (TEAC } of 
technically qualified representatives of CIA, the Department of 
Defense, USIA, and the Department of State was provided for to 
advise concerning methods, standards, priorities, etc. This 
Committee was organized on 27 January 195^* 

P . The reports of technically trained observers established 
by the cooperating agencies in a number of peripheral locations 
have been the primary raw material for evaluation. This has been 
supplemented by occasional reports from U.S. missions in the 
European Soviet Bloc c ountries, by reports from t.ha* Tfflf? . interro- 
gatlons of defectors 

^^ batisfclci 

analyses of these data are published in monthly reports. The 
Broadcast Evaluation Section takes the raw data from the monitors' 
log sheets and processes it ontOcXBM punch cards for analysis and 
recording. 

0. Under authority of NSC - 169 , technical monitors were 
established in U.S. missions behind the Iron Curtain. The project 
was established on a trial basis for thre^month^. beginning 
March 195**, under the nams of Project This project 

provided data of a different type, from which important conclusions 
were drawn. It is planned to make this project permanent where 
possible. 

®* The first prob lem encoun tered in evaluating the technical 
effectiveness of the three broadcasting organizations wa g ^ ~ 
ardizftfelnn report i ng method s. Agreement was reached on the 
definitions of good, fair, poor, and nil intelligibility. For 
most analyses, "good" and "fair" are lumped as Satisfactory, and 
’poor" and "nil" are lumped as Unsatisfactory. 


I. The problem of developing a method of summarizing and 
presenting the considerable quantity of data was given to si private 
consulting firm. Their recommendation, approved by TEAC, has been 
used. No single measure of technical efficiency was found which 
would be completely satisfactory and two measures are currently 
used. The "program effectiveness" is a measure of how many days 
a program was intelligible during the month on at least one fre - 
quency regardless of how many frequencies the program was trans- 
mitted on simultaneously. The "channel time effectiveness" reports 
the per cent of intelligible reception for observations of a 
particular program on all frequencies used and for all days of the 
reporting period. Monthly reports have been issued beginning with 
October 195** using these measures. 
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^elusions drav.n_from statistical^ analysis^©* * monitors' 
® 5 SE^ 4 angJ 5 u The 
significance of the sample", that is the indivld'ual“-jnonItoring 

l^ er +^ Pt i® not known a ® yet with respect to a great many factors; 
and, therefore, any conclusions drawn from such data must he viewed 
as tentative and with a background of full understanding of the 
circimstances and conditions under which the intercept was mad© 
and the process of analysis applied to the data. 

K„ Some of these variable factors are listed below, 

, , 1 * Monitor's Judgment of intelligibility: Since the 

wtofia t ''^ Q d" t S liglbility ** a nonltor lE ^ely subjective, 
wnat is good to one man may sometimes be "poor" to another. 

To what extent this variable factor affects the final evalua- 
tion is not known at this tin®. However, to minimize this 
error, standard tapes have been prepared in four principle 
languages with samples of intelligibility rated good, fair, 
poor, and nil by the consensus of a panel of linguists. 

Copies of these tapes will be furnished to the monitors for 
25X1 A6a us ® in standardizing their judgment. A seminar for monitors 

«+LS e1 ^ during July 1955 for the purpose of setting 

standards of reporting and monitoring procedure. Study is 
being given to possible methods of separately measuring signal 
f H+JfT'fZ* krterferenc© and relating these values directly to 
telligibility as a means of reducing the range of operator 
Judgment and thereby getting more objective data. 

Frequency and duration of sampling observations: A 
monitor cannot listen to a whole program. He must check the 

nnS«? r ^f rai f °?* a QUEibeir of Frequencies and often check a 

ThT b fL L i ? U ^ ane ° U8 pr °S rams > ® &ch on a number of frequencies. 
The length of the sample, that is the time of observation, is 

JJLJJJJI* i “^ ort£ f nc f ; Sinc ® the Jammer operations are ever 
changing, the significance from a statistical standpoint 
becomes very great. Study is being given to the optimum 
period of monitoring by an engineering consultant based on 
special test recordings of broadcast reception. 

irriiM+Ji"*? 1 ?*!!! equipEsnt Uffied ln monitoring; Study has 
Jeceiver •« ^® r ? if *° ® UCh thln « as a "typical indigenous 

f nn+« * Ve f iatol ® s d ® 8i S n » state of adjustment, type 

® tC ^ introduc ® wWe variables. It has been found 
“ 8 ® a P^telon receiver whose characteristic® 
and sensitivity are precisely known and can be standardized 
and to cross-check this against the nearest thing to a "typical 
indigenous receiver." 
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4. Area of validity of observations made at one particular 
location: Perhaps the most significant problem not yet satis- 
factorily answered is the determination of bow far from the 
monitoring post the observations may be considered representa- 
tive of reception conditions. It appears likely that no 
general rule applies and the area of validity varies from 
case to case. Certainly the area of validity is small in 
cases involving jassoing signals from near-by transmitters, 
i.e., signals transmitted by ground wave. 

* L. It has been suggested that a study of the basic factors 
involved in intelligibility might indicate methods of improving 
the intelligibility and better methods of observing and evaluating 
the Intelligibility. No work on this suggestion has yet been done. 

M. So far, analysis of monitor reports has been by program, 
that is, language and time of day. Some investigation has been 
made as to variation with frequency and transmitter location. No 
investigation has yet been made to variation with respect to type 
of program broadcast. Special methods of analysis applicable to 
standard band and long wave signals nay be fruitful, but the matter 
has not yet been fully investigated. 

N. Therefore, it may be seen that, although the organization 
has been established, procedures put into effect, a staff recruited 
and trained, and data flow initiated, results to date must generally 
be considered as tentative and subject to many qualifications. 
Progress has been and is being made in narrowing the probable 
limits of error and sharpening the significance of the conclusions 
derived from statistical treatment of monitoring data. 

II. Technical Effectiveness of U.8. International Broadcasting 


A. It is estimated th at, I n spite of th e effort put into the 

Jamming o3fu7sT internatTonal broadcasting, the" effect of the jamming 
varies band, geographic location, and target 

"language; and the programs are frequently reac hing wide sections 

of the target areas T * 

- — — — 

B. In the following paragraphs, a selection of more particular 
conclusions is presented. This may serve to indicate the kind of 
answers which the Broadcast Evaluation Section is prepared to make 
to specific questions. No attempt is made to arrive at conclusions 
regarding all possible details in view of the many variables of 
time, frequency, location, and language involved and the differing 
reliability of information concerning them. Sampling of reception 
conditions in the target area is extremely meagre both in time and 
space. The sampling on the periphery, while probably adequate in 
time, is subject to some question as to the geographic extent of 
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' appXicaoiXrcy . Moreover , the Broadcast Evaluation Section has 

investigated available information regarding medium and low frequencies 
only with respect to a few specific questions. For these reasons, 
conclusions are labelled estimates. It is expected that where more 
. reporting from within the target area becomes available, the above 
general conclusions will be confirmed, although estimates as to 
the detailed situation may be modified and extended. 


C. With respect to frequency band, as a generalisation, h igh 
frequency broadcasting is the most effective. The effectiveness 
indexes computed from the observations by the peripheral monitors 
and summarized in the monthly reports indicate that even Russian 
language programs — which receive the greatest attention by the 
jammers — are frequently receivable.* 


*For example, the summary report for the four winter months, 
November 193b through February 1955 > shows channel effectiveness 
(per cent of total observations which were classified as intelligible) 
for one or more monitoring locations as follows: 


Program 
VOA - Armenian 
VOA - Russian 

VOA - Ukrainian 

VOA - Georgian 
VOA - Latvian 
VOA - Hungarian 

VOA - Estonian 
VOA - Lithuanian 

VOA - Polish 

VOA - Czeeh-Slovak 
VOA - Rumanian 



(These figures cannot be taken too literally since they are based 
on subjective observations by a number of individuals using equip- 
ment and procedures not completely standardized and since it is as 
yet uncertain how far into the target areas observations made on 
the periphery can be applied. Nevertheless, it is believed that 
the figures are generally Indicative of the situation in adjacent 
Bloc territory in the absence of local ^ground wave/ j ammer s. ) 

Poor reception is largely, but not entirely, a reflection of the 
effect of the Soviet jaraning effort; some of the poor reception 
reported is due to vagaries of sky-wave propagation and other causes. 
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D. These reports also indicate that programs in languages of 
the USSR are generally less frequently intelligible than programs 
in Satellite languages. Persian and Turkish are the only non-Bloc 
languages that are consistently jammed. Some jamming has recently 
occurred on German language programs directed to Austria. 


£. There is certainly a considerable geographic variation 
over long distances where reception of the broadcast and jamming 
signals are by sky waves. This is apparent from considerations of 
the broadcast antenna patterns and high frequency propagation 
variations taken together with knowledge of the locations of the 
jammers used and is confirmed by the variation between the reports 
of peripheral monitors.* 

F. In addition to this large area variation, there exist 
relatively small areas near Jamming stations where steady ground 
wave signals, not subject to the vagaries of sky-wave propagation, 
will succeed In jamming a high frequency program whenever the jammer 
is on the frequency. From time to time, reports from observers 
in the target areas have given us the location of some of these 
local jamming areas. A number of reports from Moscow state that 
high frequency broadcasts are consistently jammed** and indications 
are that the jamming tra n sm i tters are located in the Moscow area. 

In Czechoslovakia, an embassy traveller reported the existence of 
high frequency jamming stations in twelve cities (ten of the 
stations specially designed for the purpose) scattered throughout 


*For exaaqale, the channel effectiveness of VOA Polish programs 
during^ the four winter months as observed at different places were 

41 
51 
35 
39 

An even more extreme example is the case of VOA Russian lan gu a g e 
programs which are as follows: 


21 
22 
2 
19 

While three locations showed remarkably consistent results, 
found only 35 intelligible samples out of a total of 1871. 

**e.g., on twenty-four days during March 1955, the embassy in 
Moscow monitored 324 VOA Russian language transmissions and 87 
transmissions of Radio Liberation. VOA was heard with good intelli- 
gibility on two occasions j Radio Liberation was not heard. 

6 
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‘ Czechoslovakia. He reports that reception in urban areas is 
generally bad; but in the rural areas , especially east of Prague, 
oL iS better. He further reported that listening to VOA and 
KFE outside the areas surrounding these twelve stations, "It is 
probable that over 75 $ of the time intelligible VOA or RFE broad- 
casts aaay be found by the patient Czech listener." 

.. * n the Far East, inf ormation until recently was derived 
entirely from the VOA monitorJMHHttpHHHMNHNNHHNMKfHI 
* reporting was begun ^^^^^^^^^^pn^ere^mw^beeiwii^sig^^^ 
nif leant reports on reception from within the target area. The 
distances from the monitor posts to the target areas are generally 
somewhat larger than in the European area. Jammers associated 
with the regular Soviet Bloc jamming system (i.e., using two -letter 
«««+ ntS rt observed jamming VOA and AC LB Russian language broad- 
ts. Unidentified jamming signals have been recently reported 
on programs directed to the Chinese mainland. 

< •• 

A review of the program and channel effectiveness 
statistics for July 1954 and from October 1954 through March 1955 
revealed few trends to indicate major changes in Communist high 
regency jamming activities during this period. A large portion 
of the variations noted could be accounted for by seasonal varia- 
tions in propagation conditions. However, jamming of VOA Georgian, 
Armenian, and Ukrainian increased; and regular jamming of VOA Persian 
ana Turkish language programs began. ~ * 

v j _^ 3 ® ru ^ e > reception on the higher high-frequency broadcast 
bands (15 and 17 me.) Is more often intelligible than on the lower 

J. The investigation of medium frequency broadcasting has 
been Carried forward by the Broadcast Evaluation Section only with 
respect to a few specific projects. 



2 * It is apparent# zdbx medium wave reception is usually 
impossible in most areas of Czechoslovakia. A member of the 
•S. Embassy staff Indicates that "medium frequencies are not 
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heard except when the Jammers have apparently broken down." 
This comment was said to apply both to RFE (719 and 854 kc.) 
and VOA (I19^kc0 broadcasts . At the same time, the VOA 
monitors report that the VOA Czech medium frequency 

programs are frequently (25 to 50# of the observations) 
intelligible at that location. 

N 3* Pew other reports have been received from within other 
target areas concerning medium wave reception. Reports of the 
peripheral monitors indicate the possibility of a wide range 
of intelligibility in different target areas* Jamming signals 
are observed on the MF transmissions of VOA in the Far East, 
including programs directed to China, but the extent and 
effectiveness of the J ammi ng has not been determined . 

K. No investigation has been made of the coverage of the VOA 
broadcasts on long wave (173 kc.), although reports of peripheral 
monitors ar^include^i^t^^mchine files of the Section. The 

r *p° r *' e ?; states that except for the area 

of Czechoslovakia immediately adjacent to the West Genaan border. 
Intelligible reception of VOA programs on 173 he. cannot be expected 
at any time. This condition applies no matter what the language 
of the broadcast. 


III. Chronology 

A. Authorization by RSC-I69 . The responsibility for estab- 
lishment of the Broadcasting Evaluation Mechanism was assigned to 
CIA by NSC-I69 on 27 October 1953* Plannings for the organization 
required for this mechanism were immediately undertaken, and the 
creation of a Broadcast Evaluation Section in the Office of 
Scientific Intelligence was confirmed by official Agency action on 
6 March 1954. * 


Planning . The plan for the Broadcast Evaluation Section 
was drafted with the cooperation and advice of representatives of 
USIA, State, and Defense. It proposed a staff of six technica l 
personnel and two clerical. The object was defined as follows: 

To assist top policy officers of this Government in determining 
the desirable extent and nature of American official and unofficial 
broadcasting operations, particularly to the Soviet bloc, in the 
light of Soviet countermeasures, as well as economic, military, 
and psychological factors, by providing a means of collating 
information from all sources on the effectiveness of of the 
several broadcasting efforts in aspects other than program content.” 
The plan provided that in its operation, the Section is to avoid 
duplication of effort by other organizations and to make use of the 
facilities of other organizations as far as feasible. 
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Staff of the Broadcast Evaluation Section . Staffing of 
the six technical positions of the Broadcast Evaluation Section 
proved difficult because of the rather specialized experience 
which was sought in prospective employees. A Section Chief was 
recruited by 14 March 1 95^, two additional personnel were trans- 
ferred from within the Agency in November December, two more 
were cleared and assigned in March and May 1955, and the sixth 
analyst is currently being cleared. Experience of these employees 

' roa< * Ca8 *' design and operation, radio parts manu- 

cturing, teaching of radio subjects, radio wave propagation 
analysis and mathematics, including statistical analysis. It is 
considered that each of these varied backgrounds will contribute 
to making a strong evaluation team. 

Dl a 2|£C* advisory committee of technically qualified 

representatives of the Department of Defense, USIA, and Department 
or State was specified to advise concerning methods, standards, 
priorities, etc. This committee was organized on 27 January 1954. 
it is called the Technical Evaluation Advisory Committee (TEAC) 
for International Broadcasting. Its charter was approved by the 
Defense, USIA, and the Department of State. It has 
met fifteen times for the purpose of approval of e valuat ion methods, 
planning special tests, improvement of reporting procedures, etc. 

Source s of information . The source of raw material for 
the evaluation has been primarily the reports of technically-trained 
observers established by the cooperating agencies in a numb er of 
peripheral locations. This is supplemented occasionally by observers 
0 , Y1A0n in u,s r “issions i n the Europ ean Soviet Bloc countries. (For an 

25X1 A2g account of Project see paragraph III G below.) In 

addition/ Information concerning effectiveness is obtained from 
other sources such as the BBC, interrogations of defectors, 
mail intercept information. 

Peripheral monitoring stations (some part-time operations, 
some with three or more observers) currently submit observations 
as follows: 4 

APPROXIMATE NUMBER 
OF REPORTS PER MONTH 

10,000 

8,000 

7,000 

5,000 

6,000 

2,500 
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M 0 HIT 0 R APPROXIMATE HUMBER 
ORGANIZATION LOCATION OF REPORTS PER MOTOR 



It will be noted that while there are many reporting locations 
peripheral to the European Bloc area, there are only two at present 
in the Far East, located in 


.. F ; PM machin e processing . It was initially intended that 
tne voluminous observations by the peripheral monitors, which was 
to be the basis of statistical reports, would be given a preliminary 
summation by the contributing organization and that the Broadcast 
Evaluation Section would combine and evaluate these* However, it 
quickly became apparent that the quantity of data er>d the number 
of ways of analyzing them was such that a centralized machine f ile 
system was necessary. Accordingly, with the approval of TEAC, the 
Broadcast Evaluation Section arranged to receive the raw reports 
from the monitors, put the data from each observation on an TF M 
card, and use sorting machinery to determine the statistics to be 
reported. This was first applied to the analysis of Project Earl 
data (see next paragraph) and to the first monthly report covering 
October observations. Monthly summaries have been published since 
then with a winter season summary (for November through February) 
incorporated in the February report. To mechanize the process 
still further, arrangements are currently being made to purchase 
teleprinter machines with tape perforators for use by certain 
peripheral monitors. The information typed by the monitor will 
then appear on the teletype tape which can be forwarded to Washington 
for automatic punching of the IBM carts. 


G. Project | 
the Broadcast Evs 



Luation 


of the results of Project 
sn tec 


r The first major evaluation project of 
fectjLoiyras the analysis and presentation 

Under NSC-169, authorization 
ical monitors in U.S. missions abroad. 


was given to establlsi 

w - — UU.0C9 .ft.'*/* Ctwj 

Responsibility for directing this operation was given to CIA. It 
was established on a trial basis for three months from 15 March to 
15 June 1954 in order to make sure that such technical reporting 
could in fact be effective. The report on this project (eventually 
published as CIA/SI 77 - 55 t classification — SECRET) gave the 
results of statistical analysis of the observations and co nc l u ded. 
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among other things, that monitoring behind the Iron Curtain in the 
cities and remote rural areas of the target countries is necessary 
for a more complete knowledge of the technical effectiveness of 
U.S. international broadcasting. It further concluded that it is 
important that these monitoring officers be able to make observations 
outside the capital cities because there was Indication that local 
jam m i n g transmitters were largely responsible for the deterioration 
of reception in the capital* From propagation considerations, 
confirmed by a few reports of reception outside the capital, it was 
recognised that where reception was degraded by the ground wave of 
near-by Jammer a, such reception could not be used as an indicator 
of reception in other areas. Pla nning for establishment of such 
monitors where feasible on a permanent duty basis is currently 
going forward. It is expected that this man can provide guidance 
to other embassy personnel who have the opportunity to sake 
intelligibility observations during the course of travels outside 
the capital. 

j 

H. Methodology development . The first problem encountered 
in evaluating the technical effectiveness of the three broadcasting 
organizations was to standardise on a reporting method. Two organi- 
zations reported five grades of intelligibility and the other used 
four. By agreement, all three adopted four grades with definitions 
as follows: 

Good - good intelligibility Is defined as easily understood. 

Fair - fair intelligibility is defined as understandable, 
hut with difficulty because of interference or 
other causes. 

Poor - poor intelligibility is defined as not being under- 
standable because of interference or other causes. 

Nil - nil means no reception at all because of interference 
or other causes. 

In analysis, this is usually further simplified by lumping the 
first two as satisfactory and the second two as unsatisfactory. 

The next problem confronting the Broadcast Evaluation Section after 
organization was to develop & method of summarizing and presenting 
the considerable quantity of data being furnished by the peripheral 
monitors. The task was given to a consulting engineering firm, an 
organization with considerable experience in monitoring procedures 
Including the problem of monitoring Soviet Jamming signals. Their 
recommendation of the use of two figures was accepted by TEAC and 
has been used in the monthly reports. These figures labelled 
"cumulative program effectiveness" and "channel time effectiveness" 
are defined in the preface of each report since they have a very 
special meaning. (It is important to understand the method of 

11 
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derivation of these figures in order to have the proper assessment 
of their meaning! they are discussed in more detail in Section V A 
of this report.) Th® consultant firm also reviewed in a pre liminar y 
way some of the problems and uncertainties in the evaluation pro- 
cedure. These are also discussed in some detail in Section IV. 

One of the problems is the question of the optimum length of a 
monitoring sample ; i »e * , how long should the observer listen to a 
particular frequency to be able to decide with reasonable accuracy 
the technical effectiveness of the program., i.e., how long and how 
often should an observer listen to a particular frequency in order 
to make best use of his monitoring time and obtain reasonably 
accurate monthly figures of technical effectiveness. A radio 
engineering consultant of Washington been employed to analyze 
special test recordings of broadcast reception with a view to 
determining the degree of accuracy which would result from observa- 
tions of various durations and occurring at various points of the 
program. This study is currently in progress. Other problems of 
methodology discussed below, both in monitori ng and reporting, have 
received tentative answers! others are yet to be investigated. 

IV. Problems of Technique 

A. Problems of reporting . 

1. Standardization of intelligibility grading . As 
indicated above, the cooperating organizations all now use 
similar reporting forms with the same definitions of intelli- 
gibility. However, this grading is necessarily a matter of 
simple judgment by the monitor! and experiments to date 
indicate that two persons listening to the same program and 

. provided with the same definitions of grades of intelligibility 
will frequently disagree, particularly when the program is 
partly degraded and not certainly "good” or "nil." It is not 
known how much of an error in the final evaluation is intro- 
duced by this variable. However, in an effort to minimize 
the problem, tapes have been prepared in four principal 
languages (Russian, Polish, Czech, and Bulgarian) with samples 
of intelligibility rated good, fair, poor, and nil by the 
consensus of a group of linguists. Copies of these tapes will 
be furnished to the monitors to use in standardizing their 
judgment. They were also used to test the variation in present 
judgment between those monitors who attended a seminar on 
monitoring procedures conducted by the Broadcast Evaluation 
Section with the cooperation of the VQA, REE, and Radio 
Liberation in Munich during July 1955* 

2. Standardization of equipment . The effect of jamming 
on a particular program can vary with variation In certain 
receiver characteristics. These characteristics in home 
broadcast receivers can vary widely depending on the design 
and state of adjustment. Home antennas will often have some 
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directional properties which might favor either the program 
or the Jammer. Some monitors have endeavored to similate 
home installations by the use of an indigenous receiver and a 
typical antenna. However, recognizing that any particular 
home receiver may or may not be typical and a stable well- 
made communications type receiver adjusted to simulate an 
average home receiver is more apt to give consistent results, 
»o©t Editors now use the Colline 5U3 receiver et 
band-width setting. It still remins to make certain that 
this adjustment in fact approximates reception on a home-type 
set and that receivers at different stations will give con- 

8 ^4 * ^ 8 °# with respect to antennas, standard! - 

zation has been recommended on a vertical whip which is least 

directional properties; and as soon as feasible, 
will be universally applied. 

3« Standardization of monitoring procedures . The length 
o time of observations, l.e., the sample, by a monitor is 
necessarily less than the length of the program since it is 
desired to check not one but all the frequencies used for the 
program and to check other programs which may be transmitted 
at the same period and receivable at the monitoring station. 
This may for some cases require listening to twenty or more 
frequencies in the course of a half hour and, if this is to 
be done by one operator, he obviously cannot listen very long 
on any one frequency. Since the intelligibility may vary ^ 
considerably in a short time during the course of a program 
because of the shifting pattern of Jammer operations or changes 
in radio wave propagation conditions, this necessity for 
sampling introduces an unknown variable in the monitoring 
results. As mentioned in paragraph HI H above, a test con- 
tinuous recording of a particular - and presumably typical - 
program as received at five monitoring stations for a period 
of two days is currently being analyzed to determine the 
deviation in reported intelligibility which would result from 
sampling in different ways. It is expected that recoaenendatlons 
can be made as to the optimum period of listening. 

, ^ ,, J All operators are instructed to "tune for the best 
intelligibility , and it is believed that possible variations 
in grading due to tuning procedure are minimized# 

Problems of operator Judgment would be eliminated if 
a procedure could be devised to measure the field intensity of 
the signal and of the interference (or their ratio) and relate 
these in some standard way to intelligibility. This approach 
is under investigation. It is not yet clear whether a 
practical solution will be obtained. 
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* B. Problems of evaluation . 

1* Statistics with different bases . So far, analysis of 
monitor reports has been by "program", i.e., language and 
time of day. Some investigation has been made as to variation 
with frequency and with transmitter location as machine pro- 
cessing and analysts' time permits. No investigation has yet 
been made as to variation with respect to type of program 
broadcast. 

2. Special methods of analysis applicable particularly 
to MF and signals will probably be fruitful. but the 
question has not yet been fully investigated. The situation 
differs from that on high frequencies in that only a few fre- 
quencies are used; and in general, the pattern of broadcasting 
and Jamming is stable. One source of information which has 

25X1X4 
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3. Basic Intelligibility factors . It has been suggested 
. that a study of the basic factors involved in Intelligibility 
might indicate 

a. methods of improving the intelligibility of 
reception in the target areas. 

b. better methods of observing and evaluating the 
intelligibility. 

Action on this suggestion awaits the couplet ion of more 
pressing problems. 

V. Interpretation of Monitors' Reports 

The approach thus far used by the Broadcast Evaluation Section 
in statistical analysis of the peripheral monitors' observations 
yields figures which must be used with understanding (their meaning 
is not simply apparent) and caution (there are a number of variables 
in addition to those listed in Section I V A, whose importance is 
not yet fully clear). 
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A. The " five figure expression " concerning a particular pro- 
gram or broadcast period (e.g., VOA Russian at 1415 GMP) observed 
over a particular period of time (e.g . , the month of January) gives 
the following in sequence: 

1. The total number of frequency units broadcast (a 
relative measure of the broadcasting effort). 

2. The total number of frequency observations (a relative 
measure of the corresponding monitoring effort). 

3* The total number of "good" or "fair" observations 
(that is, the total observations of intelligible signals). 

4. The technical efficiency of cumulative program (that 
is, the percentage of days of the month on which the program 
was intelligible on at least one frequency ). 

5* The technical efficiency of channel time (that is, 
divided by 2 expressed as a percentage). 

The reason for giving two different figures on technical efficiency 
stems from the following considerations: In nearly all cases, the 
same program is transmitted simultaneously into the target area on 
more than one frequency. It can be argued that if at least one of 
the many frequencies can be received in the target area, then the 
reception of that program can be considered technically effective. 

On the other hand, if more than one can be received, it can well 
be argued that the reception of that program is more effective 
since there is a greater chance that the listener will run across 
one of several frequencies carrying the program in the crowded and 
partially Jammed broadcast bands. The program effectiveness is a 
measure of how many days a program was Intelligible during the 
reporting period (e.g., a particular month) on at least one frequency. 
The channel time effectiveness reports merely the per cent of 
intelligible reception for observations of a particular program on 
all frequencies used and for all days of the reporting period. The 
latter figure may be unduly pessimistic as a measure of how often 
a program is receivable, but the former may be somewhat optimistic, 
particularly if it derives from the case of only one out of many 
frequencies getting through* 

B. Variables in the procedure which are recognized but have 
been only partly evaluated are: 

1. A monitor may at times check and report on frequencies 
which are not beamed in his direction, even though hearsed to 
the target area proper. While he may nevertheless hear the 
signal because of "minor lobes" of directional antennas and 
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vagaries of radio waves, the liklihood is, of course, much 
less than if he is "on the beam"" Operational procedures 
have not yet been established to account for this which may 
in some cases depress the reported technical effectiveness . 

2. The monitor may be "on the beam" of the broadcast 
signal but not receiving jamming to the same degree as the 
target area. Several attempts have been made to evaluate 
the importance of this and, so far, they have been inconclusive. 
It certainly is important when Jamming in the target area 
occurs from near-by (ground wave) jammers. Simultaneous 
observations at two different locations of distant (sky wave) 
jamming signals on a certain channel often indicate that the 
predominant Jansaer at one place at any instant is different 
from the predominant Jammer at the other. Whether this means 
that over an extended period of observation the intelligibility 
of the jammed signal would not correlate is not certain, and 
tests generally have been inconclusive because of insufficiency 
of samples or other variables (such as local jammers or unskilled 
reporters). 

3* While it is apparent that repetition of programs at 
different times during the day will improve the chances of 
that program getting through, no attempt has been made to 
take this factor into account in the reports of the Broadcast 
Evaluation Section. 
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